Multimodal integration of spontaneously produced representational co-speech gestures: an fMRI study
نویسندگان
چکیده
منابع مشابه
A multimodal database of gestures and speech
This paper describes a multimodal database which consists of image data of human gestures and corresponding speech data for the research on multimodal interaction systems. The purpose of this database is to provide an underlying foundation for research and development of multimodal interactive systems. Our primary concern in selecting utterances and gestures for inclusion in the database was to...
متن کاملCo-Production of Speech and Pointing Gestures in Clear and Perturbed Interactive Tasks: Multimodal Designation Strategies
Designation consists in attracting an interlocutor’s attention on a specific object and/or location. It is most often achieved using both speech (e.g., demonstratives) and gestures (e.g., manual pointing). This study aims at analyzing how speech and pointing gestures are co-produced in a semi-directed interactive task involving designation. 20 native speakers of French were involved in a cooper...
متن کاملMultimodal integration of fMRI, EEG, and NIRS
Multimodal integration in the field of human brain mapping has evolved from structural-functional co-registrations toward functional-functional combinations. This paper briefly reviews fMRI-EEG, fMRI-NIRS, EEG-NIRS, and fMRI-EEG-NIRS combinations. OCIS codes: Inverse problems (100.3190); Functional monitoring and imaging (170.2655); Medical and biological imaging (170.3880); Physiology (170.538...
متن کاملTemporal Symbolic Integration Applied to a Multimodal System Using Gestures and Speech
This paper presents a technical approach for temporal symbol integration aimed to be generally applicable in unimodal and multimodal user interfaces. It draws its strength from symbolic data representation and an underlying rulebased system, and is embedded in a multi-agent system. The core method for temporal integration is motivated by findings from cognitive science research. We discuss its ...
متن کاملEEG-fMRI Multimodal Integration for Epilepsy Research
Epileptologists have now at their disposal a variety of tools for investigating human brain functions. Among the technologies of non-invasive functional imaging that have flowered in the last years, two techniques became particularly popular: the electroencephalogram (EEG) which records electrical voltages from the electrodes placed on the scalp and functional magnetic resonance imaging (fMRI) ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Language, Cognition and Neuroscience
سال: 2016
ISSN: 2327-3798,2327-3801
DOI: 10.1080/23273798.2016.1245426